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ABSTRACT 



Educational research has all too often failed to be 
implemented on a large-scale basis. This paper describes the multiplier 
effect of a professional development program for teachers and for trainers in 
the United Kingdom, and how that program was developed, monitored, and 
evaluated. Cognitive Acceleration through Science Education (CASE) is a 
program for students in grades 6-9 that focuses on cognitive conflict and 
metacognition and that has a track record of long-term increases in student 
academic achievement. The CASE method involves a significant change in 
professional practice by most teachers. CASE uses a model of continuing 
professional development (CPD) that includes coaching, working with whole 
departments in schools, and integrating both practical and theoretical 
elements. The critical outcome measure in the evaluation of the CASE 
professional development program is student achievement. The paper also 
assesses the extent to which teachers actually used the CASE innovation in 
their classes (level of use) . Data were gathered from interviews with and a 
questionnaire of teacher cohorts for the years 1991-93 (over 100 teachers 
from 13 schools) and 1994-96 (88 teachers from 11 schools) . The study also 

conducted interviews with principals and heads of science departments or 
program coordinators at each school . The study found that teachers ' 
commitment and level of use were significantly greater in schools where there 
was a unity of vision between the principal and the head of the science 
department; that teachers' level of use was significantly related to the 
presence of at least one person who was deeply committed to making the 
innovation work in that school; and that there was a greater sense of staff 
ownership in schools where the science department head had adopted CASE for 
staff -development purposes or because of a belief in the value of children's 
thinking, and where there was a formal structure for regular discussion. Five 
figures and one table are included. (Contains 15 references.) (LMI) 
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Factors influencing uptake of a large scale curriculum innovation 

Philip S. Adey, King’s College London School of Education, London SE1 8WA 
philip.adey@kcl.ac.uk 

Introduction 

Educational research has been remarkably sucessful in showing how the fundamental work of 
psychologists, philosophers, and sociologists can be translated into practical classroom 
procedures which improve the immediate quality of learning and the long term effects on 
students’ life chances. This work has, however, too often been limited to lab school 
experiments conducted at a few sites and has failed to take off into large scale implementation - 
for example through adoption by school districts. Blame for this failure of the educational 
system as a whole to realise the benefits of well-researched practice may be placed on: the 
researchers, concerned with academic publication and moving on to the next new work or on 
their failure as publicists; on the conservatism of teachers and school managers; on short- 
sightedness of politicians concerned more with elections a couple of years away than with 
benefits to the society which may not show up for 10 years; or, inevitably, on lack of funding. 
At different times and in different places probably all of these accusations can be justified, and 
may be summarised as the inertia of the system, like a massive tanker whose direction cannot be 
changed by a few speed-boats of research. 

In this paper I will describe a case in which one such speedboat, by persistence over many 
years and attention to publicity and multiplier effects, is beginning systematically to shift the 
tanker of practice in one country. In particular I will focus on the multiplier effect of a 
professional development program for teachers and for trainers, and how that program was 
developed, and is monitored and evaluated. I believe that there are lessons in this case study as 
much for the methodology of evaluating staff development as for the particulars of the 
innovation described. 



The CASE program 

Cognitive Acceelration through Science Education (CASE) is a program for grade 6-9 students 
which focuses on cognitive conflict and metacognition. Special activities (Adey, Shayer, & 
Yates, 1992; Adey, Shayer, & Yates, 1995b) are delivered by teachers about once every two 
weeks, instead of a normal science lesson, over a two year period. The CASE method involves 
a significant change in professional practice by most teachers who use it, and a school which 
wishes to embark on CASE teaching is advised to contract for an extensive staff development 
program in which all science teachers will participate over the two years in which CASE is first 
taught. 

The reason that schools are willing, and often eager, to participate in the staff development 
is that CASE has a proven track record of increasing students’ academic achievement for years 
after they have participated, and across all academic subjcets. The success of CASE has been 
widely reported in the academic literature (Adey & Shayer, 1993; Adey & Shayer, 1994; Shayer 
& Adey, 1996) and also in the national press in the UK. One example of the sort of effect that 
CASE can have on gains in achievement is shown in figure 1. 



(figure 1 about here) 

The fact that we know that CASE can have a long term far transfer effect on students is 
important in the evaluation of the professional development program for its introduction. This is 
because we evaluate the professional development in terms of the effect on students’ cognitive 
growth and learning. If we were not assured that the innovation (CASE) can, under the right 
creumstances, have the effect then we would not know wthether any failure to find an effect 
was due to the staff development or to the innovation itself. 
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CASE CPD 

(In the UK, “Continuing Professional Development” and its acronym CPD have become the 
currently fashionable shorthand for what we used to call INSET, and in the US is usually 
referred to as Staff Development). 

It is well established (Joyce, Showers, & Rolheiser-Bennett, 1987) that short centre-based 
inservice courses, however well designed and delivered, have no significant and permanent 
effect on teachers’ practice. CASE CPD was designed on athe assumptions that (a) a long ‘drip- 
feed’ process would be required; (b) it is necessary for the CPD program to include elements of 
learning theory and management-of-change as well as the practical introduction to the 
techniques of teaching for cognitive acceelration; (c) a coaching element was essential, when 
tutors worked with teachers in their own classrooms; and (d) it would be necessary to work 
with whole departments in schools, not just individual teachers. Accordingly, schools who 
wished to participate in the CASE CPD program were required to agree to continue the program 
for two years, to release teachers for centre days and to allow access to tutors for coaching, and 
to involve the whole department. The cost of the whole program to the school is in the region of 
$5000. The recent devolvement of budget control from school boards to individual schools 
facilitates school Principals’ ability to make such choices for themselves. 

A summary of the complete program is shown below: 

Two year CASE-CPD program. 



Year / 
month 


In 

Centre 


In 

school 


Purpose / activities 


1 / Sept. 




1/2 day 


Meet with Principal 


1 / Sept. 


2 days 




Meet with all science teachers, outline principles, 
timetable, and commitment required. Provide plenty of 
opportunity for questions and for all to raise concerns. 

Introduction to underlying theory. Go through first 6 


1 / Sept. - 


1/2 day 


activities. The testing programme and administration of 
the pre-test. Development of individual school plans. 

Coach and/or team-teach with teachers starting 


Dec. 

1 / Jan. 


2 days 


implementation in their own classes 

Feedback from schools on progress so far. More depth 


1 / Jan. - 


1/2 day 


on theory. Next few activities. Issues around the 
management of change in the schools. 

Coach and/or team teach with teachers in their own 


June 

1/May 


3 days 


classes; Possible sessions with whole department. 
Residential conference; 1 day for CPD program 


2 / Oct. 


1 day 




participants only, 2 days to include many others. 
Sharing experiences, working on bridging, writing own 
"Thinking Science" type materials. 

Next activities. Updating school plans, further 


2/Oct. - 




2x1/2 


management issues 

Coach and/or team teach with teachers in their own 


May 




days 


classes; Possible sessions with whole department. 


2 / June 


1 day 




Post-testing, data collection. Forward plans and the 






network for continuing support. 




The first such program was started in September 1991, and we have started a new one every 
year since. We have also run specially modified programs for Trainers, who after two years 
become able to run their own CASE CPD courses, with some quality control provided by a 
network of Trainers. We are currently recruiting strongly for the 7th cohort of CASE CPD 
starting in September 1997. The King’s College CASE CPD program has reached, either 
directly or through CASE trainers who we have trained, some 10% of the secondary schools of 
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the United Kingdom. Many more schools are using the materials without participating in a full 
inservice course, but we do not have data from such schools to be able to compare with that 
from schools within our program. 

Process-product research revisited 

There are a number of levels at whch professional development programs can be evaluated. The 
most trivial (and useless) form of evlauation is the questionnaire given to participants at the end 
of an inservice day. We do not do that. At the top level, at least for a professional development 
program whose aim is to equip teachers with the skills required to raise their students’ general 
thinking ability, the most compelling evaluation is gains in pupil ability and achievement which 
can be attributed unequivocally to the professional development of the teachers. This is not a 
simple thing to do, and many have claimed that such process - product research is so difficult 
that it cannot be done, and should not even be attempted. Richardson (1994) makes the case that 
much investigation of teacher development in the past has been very instrumental, treating 
teachers like objects to be manipulated in a vain search for sets of teacher behaviours which can 
be relied upon to deliver good student learning. As a reaction to such dubious practices, the 
trend in classroom research has shifted towards ethnographic studies of classroom ecologies. 
Here we will look again at this question, and make the argument that while ethnographic studies 
have value for certain purposes, both socio-political and professional voices are quite 
reasonable in requiring some measure of outcome from investment in staff development, and 
that process-product research not only can yield useful information, but is the only approach 
which can in principle provide guidance to teachers and teacher educators on how professional 
practice might be changed to yield higher student achievement. Firstly, some of the specific 
criticisms of process-product research should be considered. 

Doyle (1977) criticises studies in which specific teacher behaviours are correlated with 
student outcomes for the idiosyncratic way in which particular behaviours are chosen for study, 
and the unwarranted assumption of causality underlying the correlation. He compares the 
process-product paradigm unfavourably with the 'classroom ecology' paradigm: 

"...the purpose of the ecological paradigm ... is to build and verify a coherent explanatory 
model of how classrooms work, a model that can be used to ask questions and interpret 
answers about teacher effectiveness" 

It is clear that ethnographic studies of classrooms can - at a cost - provide far richer accounts of 
what happens in classrooms than can simply quantitative studies (see for example, Gardner, 
1974; Tobin, 1990). But whilst such studies provide rich descriptions, it is less clear how they 
can lead to prescriptions, that is, to advice to teachers or teacher educators about ways of 
improving their practice. 

Fenstermacher (1979) also makes much of the problem of causality. He exemplifies the 
point with correlations found between, for example, the use of probing follow-up questions by 
the teacher and student achievement. He concludes that there is no way of telling from this 
correlation whether it is the nature of the questions that causes enhanced achievement, or 
whether higher achieving students provide feedback to teachers which encourages them to use 
higher level questioning techniques. Such criticism can be met by intervention studies, in which 
a teacher behaviour postulated as causally related to student achievement is specifically 
introduced, and changes in student outcomes observed. Fenstermacher's main criticism, 
however, is that process-product researchers necessarily, and unconsciously, make 
assumptions about what counts as "good" education. He claims that quantitative researchers are 
unaware that the products they strive for are no more than culturally determined norms. But 
how important is such awareness? If teachers, students, parents, university admissions tutors, 
and employers all agree that test grades are the best measures available of achievement and 
aptitude then it seems that aiming for higher grades is a perfectly respectable aim for teachers 
and teacher educators. Evaluation of inservice programs for teachers whose aims are the 
development of instruction must always, finally, look for evidence of increased student 
performance on measures which have wide popular credibility. 

A further problem with process-product research is interaction between particular teacher 
behaviours and particular learner personalities, learning styles, or context, which makes 
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generalisation of results from individual studies difficult. In an elegant study, Gardner (1974) 
showed how the use made by different pupils of a given teacher behaviour was mediated by 
personality, such that the application of a simple process-product model could easily lead to 
erroneous conclusions. Where a particular teacher characteristic at first sight appeared unrelated 
to pupil performance, deeper analysis showed that it positively affected pupils of one 
personality type, and negatively affected pupils of a different personality type. 

Brophy & Good (1986) in a thorough review of process-product research recognise all of 
these problems, and after eliminating studies which fail to meet their rather stringent criteria for 
acceptability, conclude 

"Despite the importance of the subject there has been remarkably little systematic research 

linking teacher behaviour to student achievement. A major reason for this is cost." (p.329) 

They mean, of course, the cost of thorough and well designed studies. They find, however, 
that with more sophisticated observation methods and experimental designs, some reliable 
relationships began to be established between certain teacher attitudes and behaviours (such as 
warmth, business-like manner, enthusiasm, organisation, variety, clarity, structuring 
comments, probing follow-up questions, and focus on academic activities) and students' 
achievement. They conclude that process-product research is viable, but that it is difficult and 
requires careful attention to experimental design and interpretation to make its findings valid and 
usable. 

Even if general criticisms of process-product research can be met, there remain two 
problems particular to staff development which have received less attention in the literature. The 
first is the dilution effect. An inservice staff development program can only be one of many 
influences on teachers, and a particular teacher can be only one of many influences on the 
students. The effect of one particular staff development program is likely to be so diluted in its 
effect on students as to be undetectable. 

The second is the difficulty of isolating sources of failures of an inservice programme. 
Inservice courses are often based on unsupported assumptions about what constitutes effective 
teaching and learning. The measurability of outcomes associated with such assumed good 
practice presents a problem. If you are not sure whether or not teaching method X works, in 
any sense, then evaluation of an inservice programme designed to introduce method X which 
shows no gain in pupil learning may either be because the inservice programme was poorly 
delivered, or because method X does not work. There is no way of telling which. 

Both of these problems can, in principle, be overcome: by making the staff development 
programme sufficiently extensive so that its effect is substantial, and by evaluating the methods 
being advocated separately and establishing that, at least under optimum conditions, they can 
indeed lead to enhanced student achievement. We believe that both of these conditions are met 
in the present study. The extent of the inservice has programme has already been outlined, and 
the CASE methods have been shown to be effective independently of the present INSET 
evaluation study. 

The evaluation model 

It follows from the foregoing discussion that the critical outcome measure in the evaluation of 
CASE CPD should be a measure of pupil gain. We do this routinely now by measuring the 
cognitive gains made by individual students in experimental groups compared with controls, 
and also at the class and school level by assessing the “value added” effect of the CASE 
intervention on sutudents’ achievement (see figure 1). 

As an intermediate outcome variable we have assessed the extent to which teachers are 
actually using the CASE innovation in their classes. This is described as their Level of Use. 

We are concerned, particularly, with the effect of a set of mediating variables between the 
input of participation in the program, and the intermediate and final outcome variables (LoU and 
cognitive gain). The following are postulated as mediating variables which might severally or 
together influence the extent to which participation in CASE CPD is translated into actual use of 
CASE and pupils' cognitive growth. 



1 the sense of ownership felt by each teacher of the CASE methods (SOO); 

2 perceived effectiveness of communication about the project within the school science 
department (COM); 

3 teachers' attitudes to and familiarity with the theoretical bases of CASE (THEO); 

4 a number of factors within the school management (SMI), including: 

a) management commitment 

b) unity of vision 

c) profile of CASE within and outside the school 

d) Senior management’s reasons for buying in CASE 

e) CASE co-ordinator’s reasons for promoting CASE 

f) formal communication systems within the school related to CASE 

Figure 2 illustrates the conceptual framework in which the input, mediating, and outcome 
variables are related. 

(figure 2 about here) 

Instruments for assessing these factors were: 

A Cognitive levels are assessed with Piagetian Reasoning Tasks (Shayer, Adey, & Wylam, 
1981; Shayer, Wylam, Ktichemann, & Adey, 1978), demonstrated class tasks which yield 
levels of cognitive development of individuals in a group on a scale from preoperational 
through mature formal operations. 

B Level of Use (LoU) was determined using a Level of Use scale developed by Hall & 

Loucks (1977) based on a structured interview which yields a score on a scale from 0 (is 
not using the innovation and has no intention of using it) to 6 (is not only using the 
innovation comprehensively, but has modified it to suit local conditions while retaining the 
original main features). 

C Semi-structured interviews with (i) a Principal or Deputy Principal responsible for 

curriculum or professional development, and (ii) the Head of Science Department and/or 
CASE co-ordinator in each school yield data for the SMI factors. 

D A 16 item questionnaire designed to tap variables SOO, COM, and THEO was posted to 
every teacher in the program, together with a reply-paid envelope. Follow-up phone calls 
and some individual interviews (added to the LoU interview) ensure returns in the order of 
75%. 

Validation, pre-trials, and rater cross-checking of most of these instruments have been 
described previously. The new factor in this paper is THEO. The questionnaire for the 1st 
cohort (1991-93) contained some probes about teachers understanding of metacognition and 
cognitive conflict (two of the key aspects of CASE methods), but they were open-ended and 
because of some diffidence about appearing to be testing teachers’ understanding, they were 
optional. No one answered them, for the 1996 survey, 12 Lickert-type attitude statements with 
4 possible choices (strongly agree, agree, disagree, strongly disagree) were drafted and shared 
with faculty colleagues for comment on construct validity. After revision they were added as 
subsections to one item in the questionnaire. The direction of agreement /disagreement with 
CASE type theoretical understanding varied. For example “It does not matter if pupils 
sometimes leave the class a bit confused” was scored 3 for strongly agree, and 0 for strongly 
disagree, while “The most effective way of imparting knowledge is for the teacher to talk and 
the pupils to listen” was scored 0 for strongly agree and 3 for strongly disagree. An internal 
consistency check (Cronbach alpha with Horst modification for varying ‘facilities’) showed that 
two of the subitems did not contribute to the same construct as the others, and inspection of 
their wording showed them to be ambiguous or over-strongly worded (e.g. “In each lesson, it 
is essential ...”). A THEO score was computed from the remaining 10 subitems. 
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Sample 

Two studies have been conducted so far. The first study was made of the 1st cohort of teachers 
who participated in CASE training from 1991 - 93, and a further study has been made of the 
4th, 1994 -96 cohort. Over 100 teachers from 13 schools were involved in the first study, and 
88 teachers from 1 1 schools in the second. Results from the first cohort have been reported 
previously, but will be summarised below. Data from the 4th cohort are still being processed, 
but some preliminary results can be reported. 

Results 

In the cohort 1 study (Adey, 1995) we obtained LoU data from 40 teachers, and cognitive gain 
data from 35 classes, but both measures together were only available from 18 tecahers / classes 
For this sample, the correlation between Level of Use and students’ cognitive gain was 0.61 
(pc.Ol) - this in spite of the limited range of levels of Use available from an all-CASE sample. 
The relationship is illustrated in figure 3. 

(figure 3) 

In our original research on CASE (Adey & Shayer, 1993) we showed that cognitive gain is 
directly related to subsequent gaisn in academic achievement of CASE groups compared with 
control groups. 

There was also a strong relationship (correlation 0.79) between th extent to which teachers 
communicated with one another, formally and informally, within a department and the mean 
Level of Use of CASE in that school. This relationship is illustrated in figure 4. 



(figure 4) 

Interviews with Principals and Heads of departments were analysed as follows: statements 
within transcripts of each interview which related to the factors being investigated were pasted 
into a table where the column heading was the factor under consideration, and the row labels 
were codes for the schools. Virtually all of the interview material was allocated to the table. 

Two independent judges then ranked each school for each factor, and these ranks were used as 
the “scores” for determining rank correlation coefficients between the factors extracted from the 
interviews and other varaibles. The method is described in more detail in Adey, Dillon, & 
Simon (1995a) A number of significant relaionships were found, perhaps the most important 
being: 

• COM and LoU were significatly greater in schools where there was a unity of vision 
between the Principal and the head of the science department; 

• LoU was significatly related to the presence of at least one person who was deeply 
committed to making the innovation work in that school; 

• Where the motivation of the Head of Science for adopting CASE was either for staff 
development purposes, or because of a general belief in die value of children’s thinking (as 
opposed to doing it just because of exam results, or because someoe else was paying for it), 
there was a much greater sense of ownership amongst the satff; 

• A formal structure for regular discussion of the innovation also had a significant effect on 
teachers’ sense of ownership. 

At the time of preparing this paper, we do not yet have post test data from the 4th cohort 
schools and although all LoU interviews have been conducted, analysis and verification (each 
interview rated by two raters) has not been completed. It is possible that more analysis will be 
completed by the time this paper is presented in Chicago. I do however have one new 
relationship to report from the 1996 study. A strong relationship was found between teachers’ 
THEO score ( a measure of their understanding and acceptance of the theoretical bases of 
CASE) and their sense of ownership of the method. Correlation bewteen THEO and SOO is 
0.409 (N = 60 pc.OOl). The relationship is illustrated in figure 5. 
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(figure 5) 



A weaker, but still significant, relationship was found between THEO and COM (correlation 
.314, p<.01) 



Discussion 

There are two main points which I would like to emphasise, within the context of this 
symposium: “Research into Practice”. 

The first is that we have shown that it is possible to translate a research based method into 
large scale take-up by a school system. Although at times it seems that many of the clients 
(teachers, school Principals, school boards, local government, and other funders) are 
impervious even to well-documented evidence of positive long term effects on academic 
standards, the combination of robust curriculum description, sound theoretical foundatiions, 
demonstrable long term effects, and (let’s face it) a good PR machine is finally bringing about a 
radical re-thinking of teaching methods and departmental management practice in a large and 
increasing proportion of UK schools. But it has taken nearly fifteen years. I believe it was 
President Nixon who said “Just when I start to get sick of my own voice saying the same thing 
over and over, people start to listen”. 

The second point is methodological. I believe we have established the viability of a method 
for evaluating staff development programs which combines quantitative and qualitative data 
collection and analysis in a way which will yield a detailed account of the relationship between 
key factors in teachers and schools which mediate in the effectiveness of the program. These 
factors are all, in principle, amenable to intervention, and so we will soon be in a much stronger 
position to maximise the effect of staff development on pupils’ achievement, a case for 
combined quanti system whereby the translation of a research-based teaching method into large 
scale use within a school system is being accomplished and at the same time the associated 
professional development of teachers can be systematically evaluated. 
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% at Level 6 or above, end of Grade 8 



fig 1: Value added effect of CASE training 




Mean intake level of school, start of Grade 6, as percentile 



Figure 2: Conceptual Framework of Study 



INPUT MEDIATING OUTCOME 

school variables 
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Figure 3: The relationship of Level of Use of CASE and Cognitive Gains 
(by teacher and mean class effect sizes) 




Figure 4: Relationship of Level of Use of CASE to reported level of formal and 

informal communication about CASE in the department, school means 




School mean communication score 



Figure 5: Relationship between teachers’ adoption of the theory of CASE 
teaching and their sense of ownership of the innovation. 
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